Skip to content

fix: stale HookRef in TaskProcessor, and cmf vs. bare compatibility#113

Open
tedhabeck wants to merge 11 commits into
0.1.xfrom
feat/python_plugin_compat_0.1.x
Open

fix: stale HookRef in TaskProcessor, and cmf vs. bare compatibility#113
tedhabeck wants to merge 11 commits into
0.1.xfrom
feat/python_plugin_compat_0.1.x

Conversation

@tedhabeck

@tedhabeck tedhabeck commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

On 0.1.x, declaring a plugin's manifest kind as a Python class path (e.g. package.module.ClassName) was how in-process Python plugins were declared. This PR makes such bare-FQN plugins run out-of-process by default: at install time the FQN kind is auto-converted to isolated_venv, so the plugin executes in a per-plugin virtual environment instead of in-process. The conversion requires no requirements.txt — the plugin's package is installed directly into the venv from its source.

What changed

Bare-FQN auto-conversion (default on)

  • New classify_plugin_kind() buckets a manifest kind into known / fqn / reject using purely shape-based detection (2+ dot-separated Python identifiers, final segment capitalized) — no import or venv needed.
  • convert_fqn_kind_in_place() moves the FQN into default_config.class_name and sets kind: isolated_venv. An explicit existing class_name wins (mismatches are logged).
  • Conversion runs on plugin install and plugin catalog update, and the converted form is persisted to plugin-manifest.yaml / plugins/config.yaml.
  • Unknown kinds recognized as neither a known kind nor a class-shaped FQN raise with a message naming the supported kinds.

--no-convert escape hatch

  • New flag on cpex plugin install (pypi/test-pypi/git/local) opts out of conversion, keeping the declared FQN kind so the plugin loads in-process — preserving 0.1.x behavior.
  • Also softens an unknown/unsupported kind from a hard error to a warning.

New framework surface

  • KNOWN_PLUGIN_KINDS and plugin-type constants (native, builtin, wasm, PDP, …) in constants.py.
  • PluginRegistry.get() to look up an installed plugin by name.
  • manifest_filename_for_class() util; manifests are now keyed on the full class name.

Fixes bundled in

  • Isolated worker: stale request_id — the worker reused a main()-local request_id across loop iterations, so an error raised before the next task parsed could be tagged with the previous request's id, misdelivering the error or hanging the real caller. The id is now reset per iteration. (Also hardened worker.py stdin reading: bad readline keyword, wrong Pydantic membership tests, oversized-line truncation, refactored into a tested read_task_line helper.)
  • install pkg@ wrongly skipped — the repeat-install check dropped the version constraint and compared against a possibly stale catalog entry; an explicit constraint now always proceeds.
  • Upgrade force-rebuilt every isolated_venv — the venv cache now treats a missing manifest version/hash signal as "no signal" rather than a mismatch, so pre-existing venvs aren't wiped on first run after upgrade.
  • Multi-plugin packages thrashing the venv cache — the persisted manifest is keyed on the plugin's full class name instead of the shared package root, so installing one plugin no longer invalidates a sibling's cache hash.
  • test-pypi isolated installs — now pass --extra-index-url https://pypi.org/simple/ so transitive deps (including cpex) resolve from real PyPI.

⚠️ Behavior change on upgrade

Because conversion is on by default, upgrading moves existing FQN-declared Python plugins from the in-process model to the out-of-process isolated_venv model. Install with --no-convert to keep the in-process behavior.### Checks

  • make lint passes
  • make test passes
  • CHANGELOG updated

@tedhabeck tedhabeck mentioned this pull request Jul 10, 2026
3 tasks
…s declared with a Python class path (e.g. pkg.module.ClassName) instead of a standard kind — by auto-converting them to isolated_venv plugins that install their package into the venv without requiring a requirements.txt. Work items are tagged U2/U4/U5/U6/R3 in the code comments.

Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
@tedhabeck
tedhabeck marked this pull request as ready for review July 21, 2026 01:52
Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
…rong Pydantic membership tests, and added oversized-line truncation handling with prefix-length logging, refactored into a tested read_task_line helper

Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: habeck <habeck@us.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant